I'd also like to know why pymc_extras choose to include jacobian term while pymc explicitly exlcude jacobian term in https://github.com/pymc-devs/pymc/blob/24e9f4974b2c74528b032659e80367003ad30141/pymc/tuning/starting.py#L150

We shouldn't have a flag for this, there should just be a correct answer. I believe MAP function in main PyMC is incorrect. The change of variables is not a "sampling only" thing, and the correction terms are not constant offsets that "drop out" when doing optimization. To get the correct logp expression, they need to be included.

Isn't it a question of optimizing the user defined model instead of whatever unconstraining parametrization pymc uses for logp graphs?

That is find_MAP(model_with_transforms, jacobian=False) == find_MAP(model_without_transforms)?

No because you're still transforming the variables. Maybe I misunderstand something.

Yes but you get the result you would by optimizing the constrained model. The change of variables becomes simply a trick for scipy to optimize without finding boundary issues?

Yes but you get the result you would by optimizing the constrained model. The change of variables becomes simply a trick for scipy to optimize without finding boundary issues?

I worked through a simple example on paper and I now think you are right and I am wrong. The extras implementation should have jacobian=False. We still shouldn't have a flag.

Yes but you get the result you would by optimizing the constrained model. The change of variables becomes simply a trick for scipy to optimize without finding boundary issues?

Yes, that's why I'd like to use find_MAP with jacobian=False, because it finds the true maximum point in sample space defined by users, not the transformed space, because I use it to do frequentist analysis.

The only thing I'm not certain is whether the MAP of transformed space is useful or not. Can it serve as a better starting points for NUTS? If it has some value, then maybe we should keep a flag.

You just can change the flag to False in this PR without adding an argument and we can merge it once tests pass

I use it to do frequentist analysis

I have been wanting to add a little tool to make regression table outputs from MAP, like what you get from statsmodels (probably use their reporting code directly). If you're interested in working on that I'd be happy to help :)

Thanks for spotting and fixing this @littlewu2508